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C/0 ! Abstract 

We study the renormalized area functional A in the AdS/CFT correspondence, 
i— i specifically for properly embedded minimal surfaces in convex cocompact hyperbolic 

3-manifolds (or somewhat more broadly, Poincare- Einstein spaces) . Our main results 
^"^ | include an explicit formula for the renormalized area of such a minimal surface Y as 

an integral of local geometric quantities, as well as formulae for the first and second 
variations of A which are given by integrals of global quantities over the asymptotic 
boundary loop 7 of Y . All of these formulae are also obtained for a broader class 
of nonminimal surfaces. The proper setting for the study of this functional (when 
the ambient space is hyperbolic) requires an understanding of the moduli space of all 
properly embedded minimal surfaces with smoothly embedded asymptotic boundary. 
We show that this moduli space is a smooth Banach manifold and develop a Z- valued 
degree theory for the natural map taking a minimal surface to its boundary curve. 
We characterize the nondegenerate critical points of A for minimal surfaces in H 3 , 
and finally, discuss the relationship of A to the Willmorc functional. 

(N : 

O ; 1 Introduction 

00 . 

There is an interesting nonlinear asymptotic boundary problem in which one seeks a 
I minimal submanifold in hyperbolic space with prescribed asymptotic boundary a sub- 

^ ■ manifold in the sphere at infinity. This was treated conclusively by Anderson [2], [3] in 

the early 1980's using techniques from geometric measure theory; the solutions he ob- 
tains are absolutely volume minimizing with respect to compact variations. One may also 
pose this problem when the ambient space is a convex cocompact hyperbolic manifold, 
or even more generally a conformally compact manifold (M n+1 ,g) (all definitions are 
reviewed in §2), and it is not hard to extend the existence theory to these settings. Here, 
however, we focus mostly on the special case of properly embedded minimal surfaces Y 2 
in M = H 3 /r where T is a convex cocompact subgroup (a particular case is M = HI 3 
itself), with boundary curve dY = 7 an embedded closed curve 7 C dM. Beyond Ander- 
son's aforementioned work, in this particular setting there is also a rich existence theory 
of minimal (not necessarily minimizing) surfaces of arbitrary genus by de Oliveira and 
Soret [30j . see also Coskunzer [12], [13]. 

It turns out that there is a well-defined Hadamard regularization of the area of such 
a minimal surface, and this renormalized area is our central concern. Roughly speaking, 
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our goal is to obtain a local formula for this renormalized area, i.e. one involving integrals 
of local geometric quantities, and then to use this to study the variational theory of the 
renormalized area functional A. In order to do this properly, we must study the moduli 
space of all properly embedded minimal surfaces with embedded asymptotic boundary, as 
this is the natural domain of A. This leads to a subsidiary investigation of the structure 
of these moduli spaces and the degree theory for the natural map taking a minimal 
surface to its boundary curve. We also consider the renormalized area of a larger class 
of nonminimal surfaces. We calculate the first and second variations of A; interestingly, 
these are expressed as integrals of global quantities over the boundary curve. While 
we do not touch on all aspects of this variational theory, we are able to characterize 
the nondegenerate critical points amongst surfaces in hyperbolic space, and give some 
estimates for the numerical range of A. Finally, we show the relationship of A to the 
much-studied Willmore functional W, which suggests that A is the correct conformally 
natural generalization of W to surfaces with boundary. 

There are strong motivations from the AdS/CFT correspondence in string theory for 
studying the renormalized area, and we shall explain some of these below, after describing 
the mathematical context more carefully. 

Our results about minimal surfaces parallel a number of known results about Poincare- 
Einstein (PE) spaces, so we describe these together. Let (M, g) be a PE space; this means 
that M is a manifold with boundary, g = p~ 2 g where p is a boundary defining function 
for dM and g is smooth and nondegenerate up to the boundary, and g is Einstein. There 
is a well-defined conformal class c(g) on dM, called the conformal infinity of g, which 
should be regarded as the asymptotic boundary value of g. The space of all PE metrics 
(with some fixed regularity) on the interior of a given manifold with boundary M is a 
Banach manifold, and the conformal infinity map from this to the space of conformal 
structures on dM (which also has the structure of a Banach manifold) is Fredholm of 
degree 0. These facts were proved by Anderson [5], see also Biquard [8] and Lee [25] . Most 
existence results for PE metrics are perturbative in nature, but Anderson established a 
scheme to obtain a much broader existence theory when dim M = 4 using degree theory 
[6] . One key ingredient is the properness of this conformal infinity boundary value map 
over the preimage of scalar positive conformal classes on dM. There are substantial 
technicalities in making all of this work; recent work of Chang and Yang [10] clarifies 
some of this. 

We first prove an analogous result for properly embedded minimal submanifolds: 

Theorem 1.1. Suppose that M = H 3 /r is a convex cocompact hyperbolic manifold, and 
let .Mfc(M) be the space of properly embedded minimal surfaces in M of genus k with 
asymptotic boundary curve a C 3 ' a embedded closed (but possibly disconnected) curve in 
dM. Let£ denote the space of allC' i,a closed embedded curves indM. Then both M^{M) 
and £■ are Banach manifolds, and the natural map 

Il:Mk(M) — >£ 

is a smooth proper Fredholm map of index 0. 

These properties of II imply the existence of a Z-valued degree for it, which yields 
many refinements of the existence theory for these minimal surfaces. Some consequences 
will be described in §4. 

The proof of most of this uses various well-known tools, hence this can be regarded as 
a good toy model for the corresponding result about four-dimensional Poincare-Einstein 
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spaces. Note that the use of degree theory for the boundary map of minimal surfaces goes 
back to work of Tromba [38] in the 1970's and White [12] in the 1980's, and indeed those 
papers provided some of the inspiration for Anderson's proposal to use degree theory in 
the Einstein setting. A special case of this degree theory, for genus zero surfaces, was 
developed in [12]. 

Now return to the PE setting. Assuming the conformal infinity of the PE metric g 
is sufficiently regular, then g itself has an expansion up to some order at the boundary. 
When dimM = 4, this has the form 

dx 2 + h(x) , , . 9l o, ., „ s 

g = T ^- L , h(x) ~ ho + x 2 h 2 + x 6 h 3 + . . . ; (1.1) 

ar 

here each hj is a symmetric 2-tensor on dM; in particular, ho is a metric representing 
c(g) and /13 is trace- and divergence- free with respect to ho- All other hj are determined 
in terms of these two tensors. Furthermore, x is a special boundary defining function 
naturally associated to the choice of ho- The volume form dV g has a corresponding 
expansion 

the x~ 3 and x~ l terms are absent due to the absence of the h\ term and the vanishing 
trace of /13. The volume of {x > e} is obviously finite for each e > and has an expansion 
as e \ of the form 

Vol({x>e})~^ + ^+V(M, 5 ) + .... 

The constant term in this expansion is by definition the renormalized volume of (M,g). 
The key fact, first proved by the physicists Henningson and Skenderis [22], cf. [18J for a 
careful mathematical treatment, is that this is well-defined independently of the choice of 
metric ho € The definition of renormalized volume extends to arbitrary dimensions, 
and they show that it is well defined when dim M is even; when dim M is odd, however, it 
is not well-defined and has a simple transformation law under change of representative ho- 
For simplicity here we focus on the four-dimensional case. Using the Einstein condition 
in the Gauss-Bonnet formula, Anderson [5] noted that 

S2 



V(M, g) = ^J-x(M) - \ [ \W\ 2 dV g ; (1.2) 
3 6 J M 



here W is the Weyl tensor, and the integral is convergent since \ W\ 2 is pointwise confor- 
mally invariant of weight —4. Anderson also computed a formula giving the infinitesimal 
variation of the renormalized volume in the direction of an infinitesimal Einstein defor- 
mation k: 

DV \ 9 ^) = -\f (Ko,K)dV ho , (1.3) 



in terms of the leading term in the expansion k ~ kq + xh\ + . . .. A much easier derivation 
of this formula is given in [lj. Again, this extends immediately to all even dimensions. It 
follows from this that (when n is even), HP, and indeed any convex cocompact hyperbolic 
quotient M n /T, is a critical point of V. The variational problem for renormalized volume 
remains unstudied. When dimM = 4, V is closely related to the 02 functional of the 
underlying incomplete metric on M, and there are some interesting rigidity results using 
it, see [9]. There are also several nicely geometric results about renormalized volume in 3 
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dimensions [3D], [21] (recall that it depends on some choices here, so one has not simply 
a number but rather a functional on a given conformal class of the boundary surface). 

Shortly after [22], and motivated by the same string-theoretic concerns, Graham 
and Witten [20] proved the existence of a well-defined renormalized area A for properly 
embedded minimal submanifolds Y in a PE space where the boundary of Y is also 
embedded in dM. Two dimensions is critical for minimal surfaces in roughly the same 
way that four dimensions is critical for Einstein metrics, so it is reasonable that the 
results above about renormalized volume of four-dimensional PE metrics have analogues 
for properly embedded minimal surfaces, and this is indeed true. Our second main result 
is an explicit formula for A and its first and second variations: 

Theorem 1.2. Let Y £ Ai(M) have a C 3,a embedded boundary curve 7. Then 

A(Y) = -2vr X (Y) -\ f^\k\ 2 dA, 

where k is the trace-free second fundamental form of Y ; the integral is convergent since 
\k\ 2 dA is invariant under conformal changes of the ambient metric. Furthermore, if 
is not in the spectrum of the Jacobi operator Ly (in which case we say that Y is non- 
degenerate), and cp is a Jacobi field on Y (i.e. Ly<J) = 0), which thus corresponds to a 
one-parameter family of minimal surfaces around Y , then (relative to a normalization 
which will be explained later), <j> ~ <po + x<p\ + . . . and 

DA\ Y (4>) = -?> [ (<j>) u 3 ds; 

here U3 is the coefficient of x 3 in the expansion for the function u which gives a graph 
parametrization ofY over the vertical cylinder^ x [0, xq). Furthermore, 

D 2 A\ Y (<j),<p) = ~ J 0q (Ms; 

as we explain later, this shows that the Hessian of A is represented by the Dirichlet-to- 
Neumann operator for the Jacobi operator Ly- Finally, if M = H 3 , the unique non- 
degenerate critical points of A are the totally geodesic copies of H 2 fso 7 is a round 
circle). 

We now turn to the physical precursors of all of this. Maldacena's pioneering work 
|27j proposes that in the large t'Hooft coupling regime, the expectation value of the 
Wilson loop operator corresponding to some closed loop 7 C dM should be given by 
the area of the minimal surface Y C M n+1 with asymptotic boundary 7. The papers 
[27j and [13] already point out that one must introduce an area renormalization, which 
motivated [20] . 

Quite recently, it has also been suggested ([33]> [35]) that this renormalized area 
be used to measure the entanglement entropy of a particular region in the CFT. More 
specifically, [33] (see also [23]) proposes an 'area law': for the model (H n+1 ,S n ), the 
information of a domain C S n should correspond to the region in H" +1 enclosed by a 
minimal submanifold with asymptotic boundary d£l (which need not be well-defined, of 
course, since the minimal submanifold is not unique), and in particular, the entanglement 
entropy of a domain Q C S 2 should correspond to the renormalized area of the minimal 
surface Y C H 3 with boundary 7 = dCl (see formula (1.5) in [M])- This assertion is 
checked in the lowest dimensional case n = 1 in [35], [34], and special examples are also 
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presented in [23] for n = 2 — but the validity of the assertion in higher dimensions is 
disputed in |36j. 

Motivated by these proposals, substantial effort has been devoted in several recent 
physics papers to understanding the geometric features and renormalized area of various 
simple cases of minimal surfaces in H 3 . For example, in [TJ] the authors compute the 
renormalized area of totally geodesic planes; in [23] Hirata and Takayanagi study the 
existence of minimal surfaces with two disconected circles as asymptotic boundaries and 
also estimate the renormalized area of those surfaces; Maldacena |27] studies the case of 
a rectangle where the length T of one side approaches infinity. Furthermore, Drukker- 
Gross-Ooguri, [14J and Polyakov-Rychkov, [33J have sought to check the proposed formula 
in [27] relating the expectation value of the Wilson loop in CFT with the renormalized 
area of a minimal surface in AdS. This verification involves calculating the first and 
second variations of the renormalized area functional with respect to deformations of the 
loop 7. Since those authors did not have a usable explicit formula for the renormalized 
area, their calculations required justification for dropping certain divergent terms; in 
contrast, our local formulas allow for straightforward calculations. 

Since there seems to be active and continuing interest in these proposals relating 
renormalized area with the expectation values of Wilson loop operators, the loop equation 
and to entanglement entropy, we hope that our results will facilitate further investigations 
in this area. 

Our paper is structured as follows: in section §2 we present some background ma- 
terial needed for this work, on Poincare-Einstein metrics, uniformly degenerate elliptic 
operators and embedded minimal surfaces in convex co-compact hyperbolic 3-manifolds. 
The local formula for A is proved in section §3, and certain global aspects of this func- 
tional are studied in §5. The intervening §4 develops the moduli space theory of properly 
embedded minimal surfaces. This provides the correct setting in which to derive the first 
and second variation formulas, which appears in §6. §7 characterizes the nondegenerate 
critical points of A when the ambient space is hyperbolic 3-space, H 3 . Finally, in §8 we 
discuss the relationship of A and the Willmore functional. 

The first author is very grateful to Chris Herzog, Juan Maldacena and A. M. Polyakov 
for useful conversations. The second author wishes to thank Joel Hass, Steve Kerckhoff 
and particularly Brian White for helpful conversations. 

2 Geometric and analytic preliminaries 

We now give precise definitions of the spaces and submanifolds we shall be working with 
and explain some of their properties. We also discuss some basic results about elliptic 
operators on these spaces. 

2.1 Conformally compact and Poincare-Einstein spaces and convex co- 
compact hyperbolic 3-manifolds 

A Riemannian manifold (M, g) is called conformally compact if M is the interior of a 
smooth compact manifold with boundary and g = p~ 2 g where p is a defining function 
for dM and ~g is a metric smooth and nondegenerate up to dM. Any such metric is 
complete and has sectional curvatures tending to — \dp\j(q) upon approach to any point 
q £ dM. In particular, if \dp\j is constant along dM, we say that (M,g) is asymptotically 
hyperbolic (AH). To any conformally compact metric g one may associate a conformal 
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class on dM: 



c(fiO = [p 2 9 



TdMi 



which is obviously independent of the choice of defining function p. This conformal 
equivalence class is called the conformal infinity of g. 

Any AH metric has a normal form, due to Graham and Lee |19) . Let (M,g) be an 
AH space and fix any metric ho representing the conformal class c(g). Then there is a 
unique defining function x for dM, defined in some neighborhood IA of the boundary, 
which satisfies the two conditions 

|dlogz|^ = l, g\ TdM = h , where g = x 2 g. 

The flow lines for the gradient V 9 x give a product decomposition U = [0, x$) x dM, in 
terms of which the pullback of the metric g takes the form 

dx 2 + h(x) , , . , 9 , . 

g = h{x )~ h + xhi + x 2 h 2 + . . . 2.4) 

x z 

The defining function x associated to the boundary metric ho will be called a special 
boundary defining function (bdf). 

A case of particular special interest is when (M,g) is Poincare-Einstein (PE), which 
means simply that it is both conformally compact and Einstein. These metrics were 
introduced by Fefferman and Graham [16J as a way of canonically associating a Rieman- 
nian metric on an ambient (n + l)-manifold to a conformal class on an n-manifold, with 
the goal of finding new conformal invariants on the boundary via Riemannian invariants 
of the ambient manifold. If the conformal infinity of such a g is smooth, then the family 
of tensors h(x) in (|2.4p has a complete expansion in powers of x (and also powers of 
x n logx when n = dimX is odd, n > 5). The coefficients ho and h n -\ are formally 
undetermined, but all other hj can be expressed as local differential operators applied 
to these two coefficients; it is thus natural to think of the pair (ho, h n -i) as the Cauchy 
data of g. 

In this paper we shall be primarily concerned with the three-dimensional case. If 
(M 3 ,g) is PE, then M is isometric to a convex cocompact quotient H 3 /r. (Convex co- 
compact means that T is geometrically finite and has no parabolic elements; equivalently, 
the quotient by T of the convex hull (in H 3 ) of the limit set A(r) is compact in M.) The 
Fefferman- Graham expansion for g simplifies then, and has a special form where only 
ho, h>2 and h± are nonzero, see [15] and Epstein's appendix in [31]. These coefficients 
can be calculated in terms of the metric and second fundamental form of any one of the 
level sets {x = const. }, and the special bdf x then has the property that — \ogx is the 
distance function to this level set (up to an additive constant). 



2.2 Uniformly degenerate operators 

We shall be using results about the mapping and regularity properties for elliptic oper- 
ators which are uniformly degenerate. The theory here is drawn from [28], but see also 
[25]. 

Let A be a manifold with boundary, and suppose that (x, y) is a local chart near some 
boundary point, where x is a boundary defining function and y restricts to coordinates 
along the boundary. A differential operator L is called uniformly degenerate if in any 
such chart it takes the form 

L= ^2 a jA x ^y)( xd xY(xdy) a . 

j+\a\<m 



6 



We assume that the coefficients are smooth, or at least C 2,a up to dX. There is a 
well-defined uniformly degenerate symbol 

<T m (L)(x,y,Z,ri) = ^2 a jj0l (x,y)£>rf 

j+\a\=m 

and L is elliptic in this category of objects if this symbol is invertible for all (x, y) and 
(£, rf) ^ 0. Unlike in the standard interior case, there is a further model which must be 
studied, called the normal operator, which is defined by 

N(L)= Y, a jta (0,y)(td t y(td v ) a , 

j+\a\<m 

where (t,v) are linear coordinates on the half-space x I + 1 = dimX. Finally, 
for any such operator, we define its set of indicial roots to be the values of \i for which 
Lx^ = 0{x^ +l ). (This definition must be modified slightly when L is a system.) These 
values are the roots of the indicial polynomial Ylj< m a jo(0, y)(J? , so (in the scalar case) 
there are exactly m such values. For simplicity, we now restrict to the case where the 
degree of L is 2, and list the indicial roots as fi\ and //2- 

We shall let these operators act on weighted Sobolev and Holder spaces of functions. 
By definition Hq(X) consists of functions which lie 1? along with all derivatives up to 
order k with respect to the vector fields xd x and xd y . Similarly, Aq' q denotes the Holder 
space where the derivatives and difference quotients are measured with respect to these 
same vector fields. If E is any function space, then x^E denotes the set of functions x^v 
where v £ E. 

The basic result we need is the following: 

Proposition 2.1. Let L be a uniformly degenerate operator of degree 2 on the compact 
manifold with boundary X, and suppose that L is uniformly degenerate elliptic. If N(L) : 
t^^HUdtdv) — ► t»- l l 2 L 2 {dtdv) is an isomorphism for one value of \x G Qui,^)} then 
it is an isomorphism for every \x £ (ni,fj,%), and for all such \i, 

L : x»A k +2 ' a {X) — > x>"A k ' a {X) 

is Fredholm, with nullspace contained in x^ 2 Aq Q (X) for every i. IfN(L) is only surjective 
as a map t >J "~ 1 / 2 HQ(dtdv) — » t fJ, ~ 1 / 2 L 2 (dtdv) but its nullspace is nontrivial, then L itself 
still has closed range of finite codimension, but an infinite dimensional kernel. 

The proof is contained in [28] . 

2.3 Properly embedded minimal surfaces with embedded asymptotic 
boundary 

As explained in the introduction, there is a rich existence theory for properly embed- 
ded minimal or area-minimizing surfaces in convex cocompact hyperbolic 3-manifolds. 
Something not treated in Anderson's original investigations is the boundary regularity. 
One expects that a properly embedded minimal surface Y is as regular as its asymptotic 
boundary curve 7. This problem and its generalization to higher dimensional minimal 
codimension one submanifolds was investigated by Lin [26J , Hardt and Lin [2T] and Tone- 
gawa [39J. The higher codimension case has apparently not been treated at all, but is 
in fact not so difficult using the theory of uniformly degenerate elliptic operators; we 
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shall come back to this in a later paper. In general dimensions and codimensions, if 7 
is smooth then any corresponding minimal Y with dY = 7 is polyhomogeneous at the 
boundary, i.e. has an expansion in powers of any defining function for M restricted to 
Y\ when dimY is even, only positive integer powers appear, while if dimY is odd, then 
powers of x k logx also appear; all of this is completely analogous to the situation for PE 
metrics. The case of importance here, however, is covered by the various papers cited 
above: 

Proposition 2.2. Let 7 be C k,a embedded curve in dM, where M = M 3 /T is convex 
cocompact, k <E N, < a < 1. Then Y is C°° in the interior of M and C k,a up to 

7 = FnM. 

We discuss some features of the proof in order to bring out some consequences. This 
result is local in 7, so we may as well suppose that M = H 3 and focus on the behaviour 
of Y near some fixed point p £ 7. Using the upper half-space model with coordinates 
y e M 2 , x > 0, place p at the origin and choose a local arc-length parametrization 7(5) 
for 7 (with respect to the standard Euclidean metric on IR 2 ). Let T denote the vertical 
cylinder over 7, i.e. T = {(y, x) £ M? x 1R + : y £ 7}; thus near the origin, T = {(7(5), x)}. 

Choose two smooth families of minimal hemispheres, i.e. totally geodesic copies of 
H 2 , which lie completely inside and outside of 7, respectively, and which are tangent to 
7, and let T ± be the envelopes of these families. These are smooth mean-convex surfaces 
tangent to T along 7, and it is straightforward to use them as barriers to deduce that 
Y must lie in the open set between T~ and T + . It follows that Y is vertical along 7, 
or equivalently, that its unit normal with respect to the Euclidean metric on the upper 
half-space is tangent to R 2 = {x = 0} along 7. We now write Y as a horizontal graph over 
r. More specifically, if N = N(s) is the unit normal (again with respect to the Euclidean 
metric) at a point of T, then there is a scalar function u(s, x) and a neighbourhood U of 
the origin so that 

Y n U = {F(s, x) := (7(5) + u(s, x)N(s),x) : \s\ < e, x < e}. 

The argument above implies that u(s,0) = d s u(s,0) = 0. 

The regularity of Y along 7 is equivalent to that of this function u, and the key 
point is that u is a solution of a uniformly degenerate elliptic partial differential equation 
J-{u) = corresponding to the minimality of Y, which we derive now. The function F 
induces a coordinate chart on Y; let the indices 1 and 2 refer to the s and x coordinates, 
respectively. Letting T = j'(s), then 

F s = {1- ku)T + u s N, F x =u x N + d x , 

where k is the curvature of 7. For convenience below, write w = 1 — ku. The inward 
pointing g unit normal is equal to 

V= F * XF ° =J- 1 {-u s T + wN-u x wd x ), J:= ^v? s + w 2 (l + v? x ). (2.5) 



The coefficients of the first fundamental form and its inverse are 
(9 



ii], 



w 2 + 1i 2 u x u s 
u x u s 1 + u 2 



and (g ij ) = 



1 + u x -u x u s 



u x u s w 2 + u 2 



Next, we compute that 

F ss = (w s - ku s )T + (u ss + kw)N, F sx = -ku x T + u xs N, F xx = u xx N, 
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1 

J 



11>(li ss + Kw) - U S (W S - KU S ) 
WU XS + KU X U S 



WU XS + KU x U t 
WU XX 



Finally, use the general formula kij = e^(kij + djji^g^) relating the second fundamental 
forms of Y of two conformally related metrics g = e 2< ^g. Here 4> = — log x and V is as in 
(|2.5p . so the matrix (fey) is equal to 



1 

Jx 



W(u ss + KM)) - U S (W S - KU S ) ~ 
WU XS + KU X U S — X~ 



~ 1 u x w{w 2 + u 2 s ) 



l U 2 UeW 



WU XS + KU X U S — X l U 2 .U s W 

wu xx - x~ l u x w(l + ul) 



The equation of minimality, i.e. that g l3 kij = H = 0, is 

- U S (W S - KU S )] - 2u x U s (WU XS + KU X U S ) (r> fi N, 

, ,,,2/1 , „2\Un l^-oj 



then given by the expression 



F{u) := (1 + u 2 x ) [w(u ss + kw) .,, ., 

+ «;(«; 2 + n2K a; -2^(n2+ U ; 2 (l+n2)) 



The coefficient 1/x in this last term makes this a degenerate elliptic equation. 

Assume that 7 is at least C 3 ; we compute the first few coefficients in the expansion 
of u(s,x) as x \ 0. Set u ~ U2(s)x 2 + u^(s)x 3 + . . . (since we already know that u 
vanishes to second order). Inserting this into ^{u) = yields that U2(s) = ^k(s), but 
143(5) is formally undetermined by the equation. In other words, this coefficient must 
depend globally on Y. Just as in the Feffer man- Graham expansion for PE metrics, all 
higher terms in the expansion for u are determined by 7 and U3 and their derivatives, 
so we regard (7,1*3) as the Cauchy data for the minimal surface Y. Using the unique 
continuation theorem from |29j . it is straightforward to show that if Y\ and Y2 are two 
minimal surfaces with the same Cauchy data (7,1*3) (even locally), then Y\ = Y<i- This 
global coefficient U3 plays a central role in our work. 

As a side remark for the moment, consider C 3 surfaces with boundary Y C M with 
dY C dM, which intersect dM orthogonally (this makes sense since M has a conformal 
structure). Any such Y can still be represented near the boundary as a normal graph over 
the vertical cylinder T over its boundary curve 7, and the graph function still vanishes 
to second order. It is no longer necessarily true that 1x2 
form now satisfies 



^k. The second fundamental 



1{2u 2 -k)+3u 3 + 0(x) 
-2u' 2 + 0{x 2 ) 



-2u' 2 + 0(x 2 ) 
-3u 3 + 0{x) 



note that we now have only \k\ g = 0(x) unless 2ii2 = k in which case \k\ g = 0(x 2 ). 

Finally, since the Jacobian term J = \F S x F x \ = 1 + 0(x 2 ), we see that in these 
coordinates, the area form equals 



dA 



l + 0(x 2 ) 



dsdx. 



Writing Y e = Y n {x > e}, then by definition, the renormalized area of Y is the constant 
term in the expansion 



/ dA 



length (7 e 



+ A(Y) + 0(e). 



(2.7) 



In order for this to be interesting, we must show that AiY) is well-defined, independently 
of the choice of special bdf x. This was done by Graham and Witten [20]; their key 
observation, which is particularly simple in this low-dimensional setting, is that if ho 
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and ho = e 2xo ho are two representatives of the conformal class c(g), corresponding to 
special bdf's x and x, respectively, then x = e x x, where x( x iU) = 0(x 2 ). This means 
that x = x + 0(x 2 ), and hence in the new coordinate system (s, x) on Y, one still has 
dA = x~ 2 (l + 0(x 2 )) dsdx. From this, the claim about well-definedness of A(Y) is 
immediate. 

The only property about Y needed for this argument to work is that it is at least 
C 2 and meets dM orthogonally. Thus even in this broader setting there is still a well- 
defined notion of renormalized area of Y. To maintain the distinction, we shall denote 
this extended renormalized area functional by TZ rather than A when the surface Y is 
not minimal. 

3 A formula for renormalized area 

We now express the renormalized area of a properly embedded minimal surface Y in 
M in terms of its Euler characteristic and an integral of local invariants. In fact, since 
it is not much more complicated to do so, we find an expression for the renormalized 
area when Y lies in an arbitrary Poincare-Einstein space of any dimension and is not 
necessarily minimal, but still meets dM orthogonally. 

Proposition 3.1. Let (M n+1 ,g) be a PE space and 7 C dM a C 3,a embedded curve, and 
suppose that Y 2 C M is a properly embedded minimal surface with asymptotic boundary 
7, an embedded closed curve in dM. Then the renormalized area A of Y is equal to 

A{Y) = -2vr X (y) -±J \k\ 2 dA + J W 1212 dA, (3.8) 

where k is the trace-free second fundamental form of Y and W\ 2 \ 2 is the Weyl curvature 
of g evaluated on any orthonormal basis for TY . In particular, the integrals on the right 
are convergent. If Y is any properly embedded surface which extends to be a C 2 surface 
with boundary in M intersecting the boundary orthogonally, then (with the convention 
that H = (trk)/2), the renormalized area is equal to 

A(Y) = -2ir X (Y) + ±J (2\H\ 2 - |fc| 2 ) dA + J W 1212 dA. 

Proof. We begin with some preliminary observations and calculations. 

First, denote by R%jki and (Ry)ijke the components of the curvature tensor of g and 
of the induced metric on Y, respectively. The Ricci curvature of g satisfies Rij = —ngij, 
and from the standard decomposition of the curvature tensor of an Einstein metric, the 
components of the Weyl tensor for g are given by 

Wijia = Rijkt + gikgji - gugjk- (3.9) 

Fix a point p £ Y and choose an oriented orthonormal basis {e 1 , . . . , e ra+1 } for T p M 
such that e 1 and e 2 are an oriented basis for T p Y. Now, denoting by kfj, i,j = 1,2, 
s = 3, . . . , n + 1, the components of the second fundamental form of Y at p, the Gauss- 
Codazzi equations become 

n+l 

#1212 = (Ry)i212 ~ X>11*22 " k{ 2 k{ 2 ) = {R Y )l 2 l 2 ~ \H\ 2 + -\k\ 2 . 

s=3 
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To check this last equality, simply note that for each s, k^ + fcf 2 = 2H S , so kf { = kf { + H s 
and kfj = kfj for i ^ j, and hence 



Y^s=3 (^11^22 — XL=3 ((Ml + ^ S )(^22 + -^ S ) (^12) 2 

= ElS W - l((%) 2 + (^ 2 ) 2 + 2fe 



Combined with (|3.9p . this gives 

(i?y)i2i2 + ^l 2 - \H\ 2 - Wvna = -1. (3-10) 

This equation holds at each point. The first term on the left is simply the Gauss curvature 
K of Y; for simplicity, we continue to write W\ 2 \ 2 for the third term on the left, noting 
that it is independent of orthonormal frame. 

Now integrate over Y e = Y n {x > e} to obtain 

j KdA-^j (2\H\ 2 - |fc| 2 ) dA - j W 12 i2dA = -J dA. 
By the Gauss-Bonnet theorem, since xC^e) = x(Y) f° r e small enough, 

f KdA = 2tt X (Y) - [ K e ds, 

where K e is the geodesic curvature of the boundary r y e := dY e in Y e and ds is the length 
element with respect to the metric induced by g. Altogether we get 

J dA = -2ir X (Y) + J Kt ds + ^J (2\H\ 2 - |fc| 2 ) dA + J W 1212 dA. 

To proceed further, we use the formula that as e \ 0, 

Ke ds = length{ ^ + 0(e). (3.11) 



7e 



Deferring the proof of this for a moment, using the basic definition of renormalized area 
via Hadamard regularization in (|2.7p . we find that 

A(Y) = -2tt X (Y) + lim Q J (2\H\ 2 - |fc| 2 ) dA + J W 1212 dA^j . (3.12) 

In order to show that the second and third terms on the right have limits as e \ 0, recall 
the transformation law 

k ij (e 2 *g)=e%(g), 

for the trace- free second fundamental form k (g) under the conformal change of ambi- 
ent metric from g to e 2( ^g (this is true no matter the dimension or codimension of the 
submanifold 1"). When dimy = 2, 

\k(e 2 *g)\ 2 e2 , g dA e2 , g = \k(g)\ 2 g dA g . 

Similarly, the components of the Weyl tensor transform as 

W m {e^g) = e- 2(f, W ijke (g) W 1212 (e 2 *g) dA e ^ g = W 1212 {g) dA g . 
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Thus these two potentially worrisome terms do have a limit. Similarly, even when Y is 
not minimal, by the calculations in §2, the mean curvature H is O(x), so its integral has 
a limit too. 

It remains to prove (|3.1ip . Denote by ~K e the geodesic curvature of j e with respect 
to the metric g = x 2 g and n the interior g-unit normal to dY e in Y e . Since u ~ U2X 2 + 
U3X 3 + (D(x 3+a ), it follows that n = (l + 0(x 2 ))d x + V, where g(V, d x ) = 0. Now, geodesic 
curvature also transforms nicely under conformal re-scalings: K t = e(jl t + dn log a;). Since 
g(F ss ,d x ) = at x = 0, we deduce K e = 0(e), hence K e = 1 + 0(e 2 ); recalling too that 
ds = e~ l d~s, we obtain finally 

f length ( 7e ) 

/ K e ds = h 0(e), 

as claimed. □ 

The expression ^ J Y (2H 2 — \k\ 2 ) dA is finite only when Y intersects dM orthogonally. 
Indeed, using the notation and formulas from §2.3 again, suppose that Y is written as 
a normal graph over the vertical cylinder T over the boundary curve 7, but do not 
assume that u x (s,0) = 0. Now H = 2u x (s,0) + O(x), as follows from the formulas 
H = x(if + 2du{\.ogxy\ and dyX = g{p,d x ) = u x (s,0) + 0(x). Recalling again that 
I A; 1 2 dA is conformally invariant, we see that 

I [ (2H 2 - \k\ 2 ) dA = 4KM)|2 + 0(log 6) 

does not have a limit as e \ unless u x (s,0) = 0. This is consistent with the fact that 
the definition of renormalized area A(Y) via Hadamard regularization is independent of 
choice of special bdf x only when this same condition is satisfied. 



4 The moduli spaces Mk(M) 

Fix the convex cocompact hyperbolic 3-manifold (M, g) and an integer k > 0. We define 
7Wfc(M) to be the space of all properly embedded surfaces of genus k which extend 
to M as C 3,a submanifolds with boundary and which intersect dM orthogonally, and 
Mk(M) the subspace of all such surfaces which are minimal. In this section we study 
the structure of these moduli spaces, which are the natural domains for the renormalized 
area functional, as well as some properties of the natural map II which assigns to any such 
Y its asymptotic boundary dY = 7, which is a C 3,a closed (but possibly disconnected) 
embedded curve in dM. 

It is a standard fact that the space of all C 3,a surfaces Y C M with boundary 7 
lying in dM is a Banach manifold. The space A^jt(M) defined above is clearly a closed 
submanifold (of infinite codimension). Similarly, the space £ of all C 3 ' a closed embed- 
ded (but not necessarily connected) curves 7 C dM is also a Banach manifold. The 
corresponding structure for the smaller space of minimal surfaces is also true. 

Proposition 4.1. For each k, .Mfc(M) is a Banach manifold. 

Proof. Fix any Y £ .Mfc(M) and assume for the moment that dY = 7 is actually a C°° 
embedded curve in dM. We construct a coordinate neighbourhood around Y in Aik 
which in the generic (nondegenerate) setting is modelled on a small ball around in the 
space of Jacobi fields for the minimal surface operator on Y which are C 3,a up to dY; 
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this ball in turn is identified with a small ball in the space of C 3,a normal vector fields 
along 7. We make this nondegeneracy condition explicit below. 

To set this up, let v be the unit normal (with some fixed choice of orientation) along 
Y. If <f) is any scalar function on Y which is small in C 3,a , we can define a new surface 

y o ,0 = {exp p O(p)z/(p)) : p E Y}, 

which we call a normal graph over Y. 

The mean curvature of Yq^ is computed by a nonlinear elliptic second order operator 
J-{4>). The precise expression of this operator is rather complicated, but its linearization 
has the familiar form 

D^ =Q :=L Y = A Y + \A Y \ 2 -2; 

here Ay is the second fundamental form of Y and Ay is its Laplacian with respect to 
the induced metric. 

This Jacobi operator, Ly, is an elliptic uniformly degenerate operator of order 2. Its 
normal operator is 

N{L Y ) = t 2 d 2 t + t 2 d 2 - 2 

since the second fundamental form Ay vanishes at dY; the leading (second order) term is 
just the Laplacian on the hyperbolic plane, so N(Ly) = Ajj2 — 2. The indicial roots are 
[i\ = —1, \ii = 2, hence solutions of Lyu = satisfy u ~ a(y)x~ 1 + . . . or u ~ a(y)x 2 + . . .. 
Note that since the g- and g-unit normals are related by v = xv, in the case where u blows 
up as x \ 0, the product uv = {xu)V behaves like (a(y) + 0{x))V, or in other words, 
the solutions growing at this rate are the ones which are bounded (but not blowing 
up) at x = with respect to g, and hence correspond to moving the boundary curve 
7 nontrivially. In this g normalization, the decaying Jacobi fields vanish like x 3 , which 
should be no surprise. In any case, it follows directly from self-adjointness and integration 
by parts that 

N(L Y ) : t^H^(M. 2 ;t- 2 dtdv) = t 1+ ^ H$(U 2 ;dtdu) — ► t 1+ ^ L 2 (M 2 ; dtdu) 

is invertible when /j, = 0. By Proposition [2J3 this is true for any — 1 < \i < 2 and for any 
fjL in this range, 

Ly : x^A 2 /" — » x^A°' a (4.13) 

is Fredholm of index zero. We call the minimal surface Y nondegenerate if the nullspace 
of this mapping contains only for any [i E (—1, 2); in this case, (|4.13j) is surjective. In 
general, its cokernel is canonically identified with in the following sense. First note 
that by Proposition 12.11 again, C x 2 A 2 ' a (Y) (indeed, if 7 is smooth, any u E 
is poly homogeneous, i.e. has full tangential regularity), so we may as well drop the 
subscript fj,. Next, if / E x^A^^Y) lies in the range of (|4. 13[) . / = Lyw, then obviously 
J Y uf dAy = J Y uLyw dAy = for all u E K. (Note that this integral makes sense since 
(i > —1.) However, this gives precisely the correct number of linear conditions, so this 
necessary condition is also sufficient. 

To study A^fc(M), we must consider a broader class of deformations of Y where the 
boundary curve 7 also varies. Let V = x~ x v be the unit normal to Y with respect to the 
conformally compactified metric g = x 2 g. This vector field extends smoothly to Y, and 
its restriction to 7 = dY is the unit normal to this curve in dM with respect to ho. 
Any nearby curve can be written as a normal graph 

7^, = {exp p {ip(p)N(p)) : p E 7} 
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(where now exp is with respect to ho). We now define an extension operator £ which 
assigns to any small ip a surface Y^o which is 'approximately minimal' and which has 
dY^fi = "ftp- To do this, let u be the graph function for Y over the cylinder T. We 
define a new graph function in some neighbourhood {x < e} of the boundary such 
that u^,(s,0) = ip(s), and diu^,(s,0), j = 1,2 is determined by the formal expansion of 
solutions for J 7 ; d^.u^(s,0) could be chosen freely, but we set it equal to 1^3(5). Now let 
Uip = X u ip + (1 ~~ x) u where x is a cutoff function which equals 1 near x = 0. It is not 
hard to check that JF(t/</,) € x^A 1 '" for some < fi < 2. The extension £ can be chosen 
to depend smoothly on ip. We then have that 

D£\ (YO = w 

is a function on Y which satisfies w ~ x~ x ip as x \ and Lytt; = O(x^) for some 

(0,2). 

Finally, perturb Y^q to a normal graph over it using the unit normal for Y^q an d 
as graph function any small (p £ x m Aq' q (Y). The resulting surface will be denoted Y/^, 
and we write its mean curvature as F(ip, <p). Thus if B is a small neighbourhood of the 
origin in C 3 ' a ( 7 ) x x^h^ a {Y), then 

T: B — ►z' t A£ a (y) (4.14) 

is a smooth mapping. 

A neighbourhood of Y in _Mfc(M) is identified with the space of solutions to J-(ip, (p) = 
0, and so may be studied by the implicit function theorem. Note that 

DT\ m $,$)=LY(De$) + 4>). 

When Y is nondegenerate, D2^ r |(o,o) = Ly on x^A^ a (Y) is already surjective; this yields 
the existence of a smooth map Q defined in a neighbourhood of in C 3 ' a (7) to x^K^ a (Y) 
such that F(ip,G(ip)) = 0, and so that all elements of the nullspace of T near (0,0) are 
of this form. 

In the degenerate case, we must show that by allowing ip to vary over some suitable 
finite dimensional subspace of infinitesimal deformations of 7, we can still obtain a sur- 
jective map. If this were to fail, then there would exist a nontrivial u £ K such that for 
all 4> and <j>, L Y (D£(ip) + $) _L u. Write 77 = D£(ip). Then 

= / Ly{r] + (j))u= I n ■ V '(77 + (p)u — (77 + cp)n ■ V 'u = —2 / ipUQ, 

Jy J-y Jf 

which implies that uq (the leading coefficient of x 2 in the expansion of u) is orthogonal to 
every ip, which is impossible. This proves that T is always surjective as a function of both 
(ip,(f>), and hence finally that .Mfc(M) is a smooth Banach manifold in a neighbourhood 
of Y. □ 

Proposition 4.2. The natural map 

U:M k {M) ^£{0M) 
given by n(Y) = dY is Fredholm with index 0. 

(Recall that this means that if Y € Mk(M), then Dliy is a Fredholm map from 
TyM-k(M) to T n( -y)F with the dimensions of its kernel and cokernel equal to one another.) 
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Proof. Let K denote the nullspace of Ly acting on functions of the form D£\o(ip) + <f>, 
£ x >1 A ' a (Y). If Y is nondegenerate, then K contains no elements of 
the form (0,0), so DTly is an isomorphism. If Y is degenerate, however, then the proof 
above shows that if i = &\m(KC\x ll A^ a (Y)), so that dimker£)IIy = £, then we can make 
Ly surjective by supplementing x^ 1 A^ a (Y) with an ^-dimensional space H of functions 
of the form D£o(ip) (and we may even assume that each tp is C°°). Let H' be any choice 
of complement of H in C 3,Qf (7). The implicit function function theorem shows that there 
exists a smooth map Q from H' to H © x^A^ a {Y) such that J r {^^Q{'tp)) = 0, so the 
codimension of the range of DTly is £ too. Hence ind(DIIy) = 0, as claimed. □ 

The final general result about these moduli spaces is contained in the 

Proposition 4.3. II is a proper mapping. 

Proof. We must show that if jj is a sequence of elements in £ such that 7j -» 7 in C 3 <°, 
and if Yj £ M.k{M) has dYj = jj, then (possibly after passing to a subsequence) Yj 
converges to a properly embedded minimal surface Y with genus k and 91" = 7. 

Let r be the vertical cylinder over 7, and let Uj be the horizontal graph function 
corresponding to the surface Yj. A priori, the function Uj is only defined on some vertical 
strip where x < ej. The first step is to show that €j can be chosen independently of j. 
The only thing which prevents these graphs from existing on a uniform strip would be 
if the Uj did not have a uniform gradient bound, or in other words, that there exists 
a sequence (sj,Xj) with Sj in the parameter interval for 7 and Xj \ 0, and such that 
\Vuj(sj,Xj)\ = 1, say (any positive number would do), and \Vuj(sj,Xj)\ < 1 for all s 
and for x < Xj. The gradient and norm here are with respect to the Euclidean metric. 
Perform a hyperbolic rescaling by a factor centered at the point (7(sj),0), and then 
a translation and rotation to move (sj, 0, 0) to the origin in M 2 and to make the rescaled 
curve 7j tangent to the y 1 -axis. The result is a minimal surface Yj in the upper half-space, 
defined in a ball of expanding radius tending to infinity, which passes through (0, 0, 0) in 
the boundary, and which can be expressed as a horizontal graph y 2 = Fj(y 1 ,x) over some 

large ball in the vertical (y 1 , x)-plane. By construction, ^ (0,1) = Ui<yS ^ ,x ^ • by Rolle's 
theorem, this is bounded by \d x Uj(sj,x'j)\ for some x'j < Xj, hence by construction 

|i^(0, 1)| <1 and ^(0,l) = l/x j . 

Passing to a subsequence, as j — > 00 this minimal surface converges to a complete 
minimal surface Y C H 3 whose boundary is the limit of rescalings of 7, i.e. a straight 
line, and which can be expressed as a horizontal graph y 2 = F(y 1 , x) over all of R^i x 
with 1^(0,1)1 < 1. However, by construction, the tangent space of Y is not vertical at 
the point (0,1,^(0,1)), which contradicts the fact that the unique minimal surface in 
hyperbolic space with boundary a straight line is a totally geodesic plane. 

This argument proves that the graph functions Uj are defined on a uniform interval 
[0, e], and moreover that the boundary curves at height x = e are also converging in 

C 3,a 

(in fact, in C°° by interior elliptic estimates). Notice that this already proves that no 
handles can slide off to infinity, provided the boundary curves remain uniformly smooth 
enough. Let Yj >€ = Yj n {x > e}. This is now a sequence of compact minimal surfaces 
with boundary in the convex set {x > e} C M. The proof will be finished if we can prove 
that these surfaces have a convergent subsequence. This in turn follows from the results 
of Anderson [3] and White [4T] . In order to apply their results, it suffices to show that 
the genera of the Y^ t remain bounded, which is obvious by definition, and that the areas 
of these surfaces are also bounded. This follows from the Gauss-Bonnet theorem: since 
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each Yj is minimal, its Gauss curvature satisfies K < — 1, which implies that 
Area(y i>e )< / (-K) dA = -2irx{Yj) + / Kds. 

Now, the first term is fixed, so we must show that the second term is bounded. But this 
is immediate from standard elliptic estimates applied to the graph function it,- for Yj in 
the annulus e/2 < x < 2e since e is now fixed. □ 

As explained in [40J, it is important to work with a slightly different regularity con- 
dition: we shall replace C k,a by the closure in this space of C°°. This smaller subspace 
is separable, whereas C k,a is not, so with this new regularity restriction (which we shall 
not comment on further) both A^fe(Af) and £(dM) are separable Banach manifolds. 

Using all of these facts, we may now define the degree of n by 

deg(n)= Yl (-!) n(y) > 
ren- 1 ( 7 ) 

where 7 is a regular value of n, so each Y G II -1 (7) is nondegenerate, and where n(Y) 
denotes the number of negative eigenvalues of —Ly. This degree is a well-defined invariant 
on each component of £{dM) (once we have fixed the integer k and the component of 
M.k(M) mapping to that isotopy class of boundary curves). 

For example, when M = H 3 , and 7 is any convex curve, then by the maximum 
principle, there is exactly one properly embedded minimal surface Y with dY = 7, and 
necessarily, its genus is 0. This proves that when M is the entire hyperbolic space, then 
deg (IIo) = 1 while deg (n^) = for k > 0, on the component of £ containing connected 
curves. This has some interesting consequences. For example, Anderson [3] displayed a 
connected curve which bounds a minimal surface of genus k > 0; by genericity, we can 
assume that this curve is regular for n, and since the degree equals zero, we obtain the 
existence of yet another element in .M/^HI 3 ) with boundary equal to this same curve. On 
the other hand, de Oliveira and Soret [30] construct stable properly embedded minimal 
surfaces in H 3 with arbitrary genus, where the boundary curve has any prescribed number 
of components. Here too, for any given boundary curve, we conclude the existence of at 
least one other element of Ai^ with that boundary curve which is unstable. It would be 
interesting to compute the degree of n precisely in some of these other cases. 

5 Area minimization and renormalized area 

We now investigate the role of locally area minimizing surfaces in the study of renormal- 
ized area for minimal and nonminimal properly embedded surfaces. 

5.1 Renormalized area of absolute minimizers 

Proposition 5.1. Let 7 be a C 3 '" embedded curve in dM which bounds in M. Suppose 
that Y\ and Y% are two properly embedded minimal surfaces with dY\ = dY<i = 7. If Y\ is 
area minimizing in M, then A(Y\) < AiY-z), and equality holds if and only ifYi is also 
an area minimizer. 

Proof. Fix a special boundary defining function x and set Yj tt = Yj n {x > e} and 
7j i£ = dYj >e . The functions u\ and U2 for these two surfaces agree up to order three, so 
in terms of any local coordinate s on 7, \u\(s,x) — U2(s,x)\ < Cx 3 , with corresponding 
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estimates for the first 3 derivatives. If S € denotes the region between 71 e and "y 2tt m 
{x = e}, this gives 

Area(5 e ) = 0(e). 
Recalling that Area (Y) = e _1 length (dY) + A(Y) + 0(e), we obtain 

Area (Y 2 , e ) - Area (Yi, e ) = A{Y 2 ) - A(Y X ) + 0(e), 

since dY\ = dY 2 . 

Thus if A(Yi) > A(Y 2 ), then the new (nonsmooth) surface Y[ e = Y 2 ,e U S e would 
eventually have area smaller than Y\ e . Indeed, by the inequalities above, 

Area (Y{ e ) < Area (Y 2 ,e) + C x e < Area (Y he ) + A{Y 2 ) - A(Y Y ) + C 2 e < Area (Y he ) 

for e small enough. This contradicts the fact that Y\ is area-minimizing. □ 

5.2 The renormalized area spectrum 

The result in this last subsection suggests the consideration of the set-valued function 

{A(Y) : Y G M k (M) 8Y = 7}, 

which we call the renormalized area spectrum (of degree k). Properness of the boundary 
map ensures that Sk(~f) is always a compact set. Proposition 15. II implies that if Yq is any 
absolutely area minimizing surface with 8Yq = 7, then ^(7) := A(Yq) is a lower bound 
for Sk(j) for every k. On the other hand, trivially by (|3.ip . this set is bounded above by 
— 2ttx(Y) = 2ir(2k + 1 — 2), where t is the number of components of 7. In other words, 

S k in) C \A{i), 2ir(2k +£-2)]. (5.15) 

Note furthermore that the upper limit is never attained unless there exists a totally 
geodesic minimal surface Y with dY = 7, which never happens unless 7 is a round circle. 
The lower bound is attained if and only if there exists a genus k absolute area minimizer 
with boundary 7. 

5.3 Minimizers of the extended renormalized area functional 

In section 4.2 of |23j, Hirata and Takayanagi assert that minimizers of the extended 
renormalized area functional amongst all surfaces with a given boundary are necessarily 
area minimizing surfaces with this same asymptotic boundary. Furthermore, Polyakov 
has communicated to us his suggestion (based on his work [32J) that the area-minimizers 
amongst all surfaces with a given boundary and a given genus must also minimize a 
functional that depepnds on a quadratic expression of the extrinsic curvature. Both these 
assertions follow easily from our techniques. Recall that we are denoting the extension 
of A to this setting by 1Z. 

Proposition 5.2. Let 7 be a C 3 '° closed curve in dM which bounds in M. Then the 
infimum of TZ(Y) where Y ranges over the set of all C 3,a surfaces with dY = 7 which 
intersect dM orthogonally is attained only by absolutely area-minimizing surfaces. 
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Proof. Let Y be any C 3,a surface which intersects 8M at dY = 7. We must show that if 
TZ(Y) < 1Z(Y') for all other such surfaces Y', then Y is absolutely area-minimizing. Fix 
a decreasing sequence ej \ and let Yj = Y n {x > ej} and jj = dYj. Let Yj denote an 
area minizing surface with dY- = 7^. Following the original existence proof by Anderson, 
possibly after passing to a subsequence, we may assume that 

Yj -► Y' (5.16) 

where Y' is properly embedded and area-minimizing with dY' = 7. Standard results 
imply that the convergence is in C°° in the interior, and also C 3,a up to the boundary. 
This may be proved by an argument very similar to that used in establishing properness 
in Proposition 14.31 Indeed, if the convergence were not C , we would be able to take 
a suitable rescaling and obtain a limiting surface which has boundary a straight line in 
R 2 = dM 3 but which is not a totally geodesic plane, which would be a contradiction. 
Knowing this, if the convergence were not C 3 ' a , we could again rescale and extract a 
limiting surface which converges to a totally geodesic plane, but not smoothly, which 
also contradicts standard convergence results for minimal surfaces. 

Denote the genus of Y' by k, so Y 1 minimizes A in U S A4 S (M) amongst all minimal 
surfaces with the same boundary and of arbitrary genus. 

We now claim that 

Area (Yj) > Area (Y') = — length^) + A(Y') + o(l). (5.17) 

e j 

The first inequality is by definition. As for the second equality, we proceed in steps. Since 
Y! — ► Y' in C 3 '°, there exists p > so that each of the annuli Yj \ Yj := Yj n {x < p} 
and Y' \Yp := Y' f] {x < p} is a normal graph over some portion of the vertical cylinder 
T of height p above 7. In particular, suppose that Yj \ Y- p is the graph of a function Uj, 
defined on some band Tj = {ej < x < p] and Y' \ Y' is the graph of a function u defined 
on the band {0 < x < p}. Clearly the areas of the portions where x > p converge, i.e. 
Area (Yj ) — > Area (Yj,). Thus if we denote by Y'j, p, ej the portion of Y' that lies below 
the hyperplane x = p and above the hyperplane x = ej it suffices to show that 

hm (Area (Yj^. - Area (Yj \ Yjj) = 0. 

This follows by direct computation. First write Uj(x, s) = x 2 Uj(x, s), u(x, s) = x 2 u(x, s), 
so that Uj — > u in C . Denote by J u , J Uj the Jacobians for parametrizations of these 
surfaces in the (s,x) coordinate charts as in §2.31 Using the formulae from that section 
we compute that, with constants independent of j, 

|(Area(y; iP)£ .-Area(y/\y; jP )) 

<C / — k / In 2 — u 2 s — ((1 - ku) 2 (1 + u 2 x ) - (1 - Ku 2 j)(l + u 2 j>x )) \ dsdx (5.18) 

< C j I \uj — u\ + \uj, s — u s\ + \uj. x — u x \ dsdx — > 

as j — > 00. This proves (|5.17|) . Subtracting e~ 1 length (7) from (|5.17|) and passing to the 
limit gives 

Tl(Y) > K{Y') =A(Y'). 
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The second step of the proof is to show that if 1Z(Y) = 1Z(Y') then Y must be an area- 
minimizer. By Proposition 15.11 it suffices to show that Y is minimal. If it is not, then its 
mean curvature H is nonvanishing in some open set. We can then perturb Y locally, in 
some small ball Y f]B r (p), to a new surface Y" which is also smooth, but has smaller area 
in this ball. Clearly the areas of the truncations to x > e satisfy Area (Y") < Area {YD 
for e small, which in turn implies that 1Z(Y) < 1Z(Y'), contrary to what we proved above. 

□ 



6 First and second variations of renormalized area 

We now begin the variational analysis of the renormalized area functional A on each of the 
moduli spaces Aif.(M), as well as for its extension to the unconstrained spaces A4&(M). 
The first variation formula for A on M.^ is formally analogous to the corresponding 
first variation formula for the renormalized volume of Poincare-Einstein metrics in even 
dimensions; this formula appears in a paper by Anderson [5], see also Albin [1| for a 
simpler approach. A formula for the second variation of renormalized volume does not 
seem to have been computed in the Poincare-Einstein setting. 



6.1 The first variation 



We shall compute the first variation of A at any Y £ Aik(M); slightly more generally, we 
compute the first variation of the extended functional 1Z at any Y £ M&(M). Actually, 
we compute D1Z only applied to compactly_supported perturbations, and show this is 
not well-defined for arbitrary variations in M^{M). 

As before, fix a special boundary defining function x on M, and for any Y £ M.^{M) 
let 113 denote the free third order term in the expansion of the graph function u of Y 
with respect to x over its vertical cylinder. 

Theorem 6.1. Fix any Y £ Aik(M) with dY = 7. Let Yt denote a smooth one- 
parameter curve in A4k(M), dY t = "ft, with Yq = Y (so the surface Yq is minimal but the 
surfaces Yt, t ^ 0, need not be). Write Yt as a normal graph Y^ over Yq = Y via 



Y t = {F t {p) = exp p (&(p)i/(p)) : p £ Y} 



where (f>o = 0. 
Setting <$> - 



then 



+ 



d_ 



AYt) 



t=o 



On the other hand, if Y £ A^fc(M) and 
then: 

d 



dt 



AY) 



., and we have 
4> U3 ds. 

outside some compact set K C Y , 
H<j>dA 



t=o 



Proof. By virtue of (|3.8p . we have 



d 
li 



t=o 



A ^ = 2 It 



t=Q 



y, 



\h\ 2 dA t 



(6.19) 



where kt and dAt denote the second fundamental form and area form on Yt (tr kt denotes 
the trace of the second fundamental form of Yt). 
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In order to compute this, introduce the following notation. Fix any point p GY and 
choose an orthonormal moving frame {ei, e<{\ on Y near p, as well as a unit normal vector 
field v, so that e\ and e2 are the principal directions and V e ^j = —Ki<>ijV at p (the are 
the principal curvatures of Y). Define r/i(t) = (F t )*(ei), so r/j(0) = but {771 (t) , 772 (^)j- is 
no longer orthonormal when t 7^ 0. We define 

9ij(t) = (vi(t),Vj(t)) and = -(V^rjjit), u(t)), 

where v{t) is the unit vector orthogonal to both r]i(t) and 772 (i) with z/(0) = v = e 3 . 
Finally, write T = F^dt, so T = (j)v when t = 0. 

In the following, all terms are to be computed eventually at p. We first compute that 

^TVi\ t =o = v TVr ;i F| t=0 = V ei Vr-F| t=0 = V 6i (j>v) = <pHiei + fav, 

which yields immediately 

T 9ij(t)\ t=0 = (4>Kiei + <piv, ej) + (a, (pKjej + fav) = 2<pkij, 

since fey = «j<5y in this frame. This also implies that Tg^(t)\t=Q = —2(j)k % K It is also 
standard that 

^ F*{dA t ) = <j>(tTk ij )dA . 
at t=Q 

Finally, 

By assumption, V ei ej is orthogonal to y, while V v v is tangential, so the second term on 
the right vanishes. By definition of the curvature tensor, the other term equals 

-(V^Vt^- + V[T,m}Vj + R(T, r)i)Vj,") = + <t>j u )i u ) ~ 4>{R{v, ei)ej,v). 

Observe that 

[T, iji] = V T ei - V ei (4>u) = + fcv - fai* - (fi^a = 0, 
so expanding out yields that 

Tk ij\t=0 = ^Kpij ~ faj + R 3i3j = -(^ 2 4>)ij + <P(k O k)ij + R 3 i3j- 

Putting these formulae together, we compute that 
d 



dt 
+ 



- J (tr k t ) 2 - \k t \ 2 dA t = - J (-V 2 ^) k ij + Aj>tr kdA 
J (tr kf - 3|£f(tr k) + 2tr (k o k o k) dA + 2 ^ <HiWtr k - R 3i3j k ij ) dA. 



(6.20) 



(Here tr (kokok) = g %v 'gi q g tr ki r k p jk q £.) A simple calculation using the principal curvature 
decomposition shows that 

(tr/c) 3 -3|/c| 2 (tr/e)+2tr(/co/co/c) =0, 
while since M is hyperbolic, 

iW tr k - R 3i3j k ij = -tr k. 
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This proves 
d 



dt 



- [ (tr k t ) 2 - \k t \ 2 dA t = - [ -V 2 <pijk ij + A4>trkdA + 2 [ 4>trkdA. (6.21) 
2 JY t Jy Jy 



t=0 JYt 



The next step is to evaluate J Y ^— V 2 ^/^ _|_ A</>tr kj dA. To do this, we integrate 
by parts on Y e = Y n {x > e} to get 



I, 



((V 2 <p,k) -A<£trfc) dA = 

J &{V j k ij -V i k j j )dA + j ({V<p,n)trk-k(V<p,n)^j ds, (6.22) 

where j e = dY e and n is the g- unit normal in Y e to r y € . The contracted Codazzi equation 
states that 

V j kij = Vjtr k + Ric^, 

but in fact Ric^j = —2g u ; L = since the i index refers to a vector tangent to Y. Hence 
only the boundary terms remain. If Y € Mfc(M) then since </> is compactly supported, 
these boundary terms vanish. If Y € Mk(M), however, then tr k = so the first part of 
the integrand in the boundary integral vanishes and we only have to evaluate the second 
one. 

To do this, revert to the (s, x) coordinates introduced in §2. In terms of these, the 
expression becomes 

kg tj kjen e ds. 



7e 



To calculate this, we first note that the unit normal n = n 1 ^ + n 2 d x has coefficients 
which satisfy 

n 1 = 0{x 3 ), n 2 = l + 0{x 2 ). 

Thus we may as well set £ = 2 and n 2 = 1. Note also that g 1 ^ = x 2 ^ and ds = 
x^ 1 ds, where s is arclength on 7 with respect to g. (This differs slightly from our earlier 
convention.) The terms in the integrand are thus 

(<j>ix 2 (g n k 12 +g 12 k 22 ) + <j> 2 x 2 (g 21 k 12 + g 22 k 22 )) ds. 

Finally, recall that (ft ~ x~Vo and use the expansions for the kij and gij to deduce that 
this reduces to 

-3 / 4> u 3 ds + O(e). 

This completes the proof. □ 

From this formula we deduce the 

Corollary 6.1. Suppose that Y £ Mk(M) is a critical point for A. If Y is a nonde- 
generate element in Mk (i.e. it has no decaying Jacobi fields), then the coefficient U3 in 
its boundary expansion vanishes identically. In general, ifY is degenerate, then we may 
only conclude that 113 is orthogonal to the finite dimensional space spanned by leading 
coefficients ipo of Jacobi fields ip ~ tpox -1 + ipix + . . . along Y. This is equivalent to 
the statement that U3 lies in the (finite dimensional) span of all leading coefficients tps of 
decaying Jacobi fields ip ~ 1P3X 2 + . . . onY. 

IfYG Mfe(M) is critical for 1Z, then Y is necessarily minimal, and U3 = regardless 
of whether or not Y is degenerate in J\4k(M). 
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We remark that it would be quite interesting to know if there exist degenerate critical 
points of A. 



6.2 The second variation 

We now derive the second variation formula for A at a general element Y £ Aik- The 
calculation is more involved than for the first variation formula, but is actually fairly 
elementary since it relies primarily on some detailed trigonometric calculations. 

Before stating the result, we set up the basic notation. We use the same language as in 
Theorem l6.11 Let Yt be any 1-parameter family of surfaces in A^fc(M), with corresponding 
family of boundary curves 7$ in dM. The calculations are local, so we assume that 
M = B 3 and we work in standard upper half-space coordinates. Let Nt denote the 
inward-pointing unit normal to 74 with respect to the Euclidean metric on M 2 . Let (s,v) 
be Fermi coordinates around 70, i.e. corresponding to the map (s,v) 1— ► 7o(s) + vNq(s). 
Then jt can be written as a normal graph, v = ipt(s), where ipo = and ipN® is the 
normal variation vector field. Similarly, introduce Fermi coordinates for the Euclidean 
metric around Yq, using the inward unit normal 17; the cylindrical coordinates (s,x) 
parametrize Yq, and we write Y t as a normal graph v = (f>t(s,x) using the (/-unit normal 
V. The variation vector field is 017, so is a solution of the Jacobi equation Ly (p = 0. 
The regularity theory for this equation shows that it has an expansion 

0(s, x) ~ O (s) + <j)2(s)x 2 + 3 (s)x 3 + . . . , 

where 0o(s) = ip{s). If Yq is nondegenerate, the coefficient of x 3 is determined by a global 
Dirichlet-to-Neumann map in terms of the leading coefficient, which we write as 

1 o I 

^3 = g d x \ x=Q (f) = VY (t>o- 
Theorem 6.2. The second variation of A along the family Yt is given by 

D 2 A\ Yo (<j>J) = -3 f 0o03*, 

or equivalently, when Yq is nondegenerate, 

-3 / o ry o 0oc?s. 
J 70 

Proof. Let us begin by setting up some notation. Write each Yj as a horizontal graph 
over the vertical cylinder T t on the curve 74, i.e. via (7t(s), x) + ut(s, x)N t . Through the 
remainder of this proof, subscripts refer only to t-dependence but not derivatives, and 
we often write w for dtw. Set 

u t (s, x) = ^K t {s)x 2 + u^ t {s)x 3 + 0(x 3+a ). 

The unit tangent and normal vectors of jt are 
^ 1 



^(ds4> t ) 2 + (1 - ko0 4 ; 



((1 - K O t )T o + d s (/>tN ) 



\/{o 8 9tr + l 1 - K o0t) 2 
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so the component along N t of the variation vector field 4>Nq is 

(N t ,0 o ) = k \~ K ^ ^^ = j> t (i + - K0 <Ptfy 1/2 

\J{d s <p t y + (1 - nofaY 



This is equal to ^(1 + 0(t 2 )). Hence the first variation of A at Yt is given by 

-A(Y t ) = -3 [ u u fa( l + 0(t 2 ))ds. 

J-y, 



d_ 
dt' 



Differentiating once more at t = yields 



= ~ 3 / ( ^3,0 4>0 + ""3,0^0 ) ds, 
t=0 J 70 



The second term in the integrand is an artifact of the parametrization. Indeed, recall that 
for the composition of any two functions, A{s) and s = Y(t), (A(Y(t))" = A"(Y )(Y') 2 + 
A'(Yq)Y" , but the Hessian of A corresponds only to the first term on the right. In other 
words, 

D 2 A\ Yq (</>, j>) = -3 / n 3 ,o0o ds. (6.23) 



To 



The key point is to relate ^3 to ^3^. The long calculation that follows involves 
expressing 4> t in terms of ut and then calculating dtd% at (x,t) = (0,0). The relationship 
between these two terms arises as follows. Fix any P £ Yq with x coordinate x{P) = xq 
sufficiently small. Let A be the line emanating from P G Yq in the direction v{P). Denote 
by P and R its points of intersection with Yt and Y^, respectively. Note that assuming 70 
is convex near P, then moving from P along the line A, one encounters first P then R; 
we orient coordinates so that all quantities are positive in this situation. By definition, 
4>t(P) is the distance from Y to Y t along A, and it can be decomposed as the sum of two 
signed distances, from P to P and from P to R, which we call £ and x, respectively: 

fa = \PR\ = \PP\ + \PR\ =I+X- 

We show how to relate each of the two quantities t and x to ut in turn. 

First consider I. Let xq be the x coordinate of the point P and a the angle between 
A and the plane x = xq. From (|2.5p the orthogonal projection of T>(P) onto this plane is 
spanned by —8 s uTq + (1 — kqUq)Nq, and 

d x u (1 - KqUq) 2 3 

sin a = — . =^ a = KQX + 6u3nx + C (x ). 

^ {dgU) 2 + (1 _ Kono) 2 (1 + (^2) 

Now let A 1 denote the line in the plane x = xq from P in the direction of this projection 
of 17, and P 1 = A 1 n T t . Setting l l = \PP 1 \ (interpreted ctS 3> SI gned distance), since 
PP 1 P is a right triangle, we have 



I 1 



I 



1 (l + \a 2 + ...)=i 1 {l + \k 2 x 2 + 3k u 3i0 x 3 + ...) 



cos a 2 2 

We shall prove below that 



? 1 L =0 = ^ 5 ^1^0 = 0, and d^^e^o + O^ 2 ), (6.24) 
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and hence, after some calculation, 



dtdll 



= 18k «3,o<Ao- (6.25) 

x=t=0 



To prove (|6.24p . we examine £ more closely. Consider the (very thin) triangle T in the 
plane x = x$ with sides the segments of the line A 1 , the line A emanating from P with 
direction iV~o, and the line tangent to 74 at the point of intersection AD 74. Let £ 2 , £ and £ 
denote the lengths of these sides, respectively, and let (3 and uj denote the angles between 
A 1 n A and A D j t - Note that 

cos/3 = 1 (-d s uT + (1 - k u )N , N ) 

V {d s U )< ! + (l-K U )' ! 

= (1 + (d s U /(l - K U )) 2 y 1/2 = 1 - ±(±C> s Ko X 2 + d s U 3fi X 3 + ...) 2 + ... 

=>P= ^d s K x 2 + d s u 3 , x 3 + .... (6.26) 
In addition, since ir/2 — uj is the angle between Nq and N t , 

cob(| - u) = 1 - ~(a 8 & + o{t 2 )) 2 + ...=}► w = I - a s< /» t + o(t 2 ). 

The law of sines for this triangle is the set of equalities 

£ £ 2 £ 



sin/3 sin a; sin(7r — (3 — u) 

Since uj is bounded away from 0, the first equality and (|6.26p imply that £ = 0(x 2 ), 
which in turn shows that £ 2 = £ x + 0(x 4 ). Thus for computing third derivatives, we may 
as well work with £ 2 . The second equality yields 

e = £ Sin ^ = J— (1 - tan /3 cot ^ + 0(x 4 )) . 

sm(7r — p — uj) cos p 

Recalling that £ = uq + ipt, and using the expressions for (3 and uj above, we obtain 

£ 2 = ^t + u Q ){i + o(x i )){i-(\d s K Q x 2 + ...){d s 4> t + o{t 2 )) 

= Vt + ^ox 2 + u 3i0 x 3 + 0(t 2 ) + 0(x 4 ), 
which yields (^24"]) . 

We turn to the computation of \. The idea is much the same. Let (so, xo) denote the 
cylindrical coordinates of P, i.e. as a horizontal graph over the cylinder To, and (s' ,x' ) 
the cylindrical coordinates of the point R = A Pi If, when written as a horizontal graph 
over T t . Since R = P + 4>t{s, x)v and 17 = /Vo — KQxd x + 0(x 2 ), we obtain 

s' = s + O(x 2 ), x' Q = x - Koxfa + 0(x 2 ). (6.27) 

We shall compute the horizontal displacement ut(s' , x' ) of Y t from T t in the plane x = x' . 
For this, let j t = T t n {x = x' }, which is just the vertical translate of 74 into this 
plane. Abusing notation slightly, write P for the point on jf which is directly below the 
intersection of A Pi Tt (which had been called P above). Finally, let Q denote the point 
on 74 which is nearest to R, i.e. so that R = Q + cNt for some constant c. 
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Consider the thin triangle PQR. The sides PR and RQ have lengths xcosa an d 
ut(s' ,x' ), respectively. Let r\ and ( be the angles at the vertices P and Q. We find that 

V = l + 0(t) + 0(x 2 ), ( = l + 0(x 2 ). 
By the law of sines again, ut(s' ,x' )/ sin 77 = xl so 

Xcosa = u t (s' ,x' )(l + 0(x 4 ))(l + 0((t + x 2 ) 2 )) = u t (44) + 0(t 2 ) + 0(x 4 ). 
Expanding in a Taylor series, using H6.27H and recalling that ut = 0(x 2 ), we find that 

u t (s' ,x' ) = u t (s ,x ) + d s u t (s ,x )O(x 2 ) + d x u t (s ,x )(-K x(pt + 0{x 2 )) + 0(t 2 + x 4 ) 
= u*(s , ^o) - K K t (p t x 2 - 3nou 3tt <f) t x 3 + 0(t 2 + x 4 ) 
= (jjKt - noHt4>t)x 2 + (n 3jt - 3K ti3,t<^t)2; 3 + . . . ; 

also, as before, 1/ cos a = 1 + 0(x 2 ), so altogether we obtain 

dtdlx\ t=x=0 = 6n 3j0 - I^kqu^ 



We have now proved that 

dtdl4>t\ „_,_n = d t dl(l+ x) = 6Py o (0o) = 6-u 3i0 + 18^07*3,0^. 

Inserting this into (|6.23|) yields the formute in the statement of the theorem. □ 



7 Critical points of A in H 3 

In this section we prove that the only nondegenerate critical points of renormalized area 
for proper minimal surfaces in all of H 3 are the totally geodesic planes, the boundary 
curves of which are circles. The proof requires a preliminary geometric lemma about 
osculating circles of plane curves, which is perhaps of independent interest, and then 
proceeds via a refined version of the asymptotic maximum principle. 

Recall that the osculating circle C at a point p G 7 is a circle which makes second 
order contact with 7 at that point; its curvature, the inverse of its radius, is therefore 
the same as that for 7 at this point of intersection. By inscribed we simply mean that C 
remains entirely within the closure of one of the two components of S 2 \ 7. 

Proposition 7.1. Let 7 be a C 2 embedded loop in S 2 = dM 3 and one component of 
S 2 \ 7. Then there exists a point p G 7 so that the osculating curve of 7 at p lies in Q. 

Remark 7.1. As the proof makes clear, there may be many such points. Of course, we 
could equally well have replaced Q by the other component of S 2 \ 7. 

Proof. We begin with a few elementary observations. First, the entire question is invari- 
ant under Mobius transformations, hence we may freely apply such transformations to 
reduce the problem to one that is easier to visualize. Second, if C is any circle inscribed 
in Q which is locally maximal (in the sense that for any continuous family of inscribed 
circles C"(e) with C"(0) = C, the family of radii r'(e) reaches a local maximum at e = 0) 
then necessarily either C is tangent to 7 at two or more distinct points, or else C is 
tangent to 7 at a single point and is the osculating circle there. The reason is that if there 
is only one point of contact, p = C fl 7, then the curvature of the circle 1/r' is greater 
than or equal to n(p), the curvature of 7 at p. If this inequality is strict and there are 
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no other points of contact, then we could increase the radius of C slightly while keeping 
it inside f2. 

Thirdly, and slightly more complicated, we claim that if C arises as a limit of inscribed 
circles Cj such that each CjCij contains at least two points Pj ^ Qj and dist (Pj, Qj) — > 
as j — > oo, then the limit C is necessarily an inscribed osculating circle. To see this, 
choose for each j a Mobius transformation Fj which carries Cj into a fixed straight line 
in M 2 , say the yi-axis. We also suppose that dist (Fj(Pj), Fj(Qj)) = dist (Pj, Qj), so that 
Fj does not diverge, that Fj carries Q to the lower half-plane, and that limFj(Pj) = 
lim Fj(Qj) is the origin. Each curve Fj(j) lies in the upper half-plane and is tangent to 
the ?/i-axis at two points which are converging to the origin. Clearly, the curve must be 
a graph over the axis between these two points for j large enough, say of some function 
fj. By the intermediate value theorem, there is a sequence of points tj — * such that 
fj(tj) = 0. Taking a limit, we see that the limiting curve is flat to second order at the 
origin and lies entirely in the closed upper half-plane, which proves the claim. 

We can now proceed with the proof. Let K, denote the set of pairs (P, Q) G 7 x 7, 
P 7^ Q such that there is an inscribed circle which is tangent to 7 at these two points 
(and possibly other points as well). By the second remark above, if this set were empty, 
then there would have to be an inscribed osculating circle already. So assume 1C 7^ 0. We 
claim that the closure of K, must intersect the diagonal, which by the third remark above 
would produce an inscribed osculating circle: If it did not, then /C would be compact in 
7x7, and hence there would be a point (P' , Q') = (7(^1), 7(^2)) such that the difference 
in parameter values | ti — t\ | is minimal (for some fixed parametrization of the curve) . Let 
C be the corresponding circle. Conformally transform so that C is the yi-axis. Then 
the images of P' and Q' lie on this axis and the transformed curve lies entirely on or 
above the axis. If the image of P' is the left-most point of tangency of the curve with 
the yi-axis, fix another point P" just to the right for which the horizontal component of 
the downward pointing normal is positive. There is a maximal radius for which a circle 
tangent to the curve at P" remains in the component of the lower half-plane, and this 
circle is obviously tangent to the curve at another point Q" to the left of the image of Q' . 
This shows the existence of another pair (P",Q") strictly between the pair (P',Q') for 
which there is an inscribed circle tangent at these two points, which is a contradiction. 
This finishes the proof. □ 

Now we turn to the main result of this section. 

Theorem 7.1. Let Y £ _Mfc(IHI 3 ) be nondegenerate and suppose that dY = 7 is C 3 ' a and 
connected, and Y is a critical point for A. Then dY is a round circle and Y is a totally 
geodesic disk. 

Remark 7.2. Using Corollary \ 6.1\ the proof below actually shows that if Y is critical 
for the extended renormalized area functional 1Z, then the same conclusion holds. 

Proof. First, by the results in the previous section, we know that Y must be a minimal 
surface and also that the formally undetermined term U3 in its expansion must vanish. 

Using the result about osculating circles, and applying a conformal transformation, 
we reduce to the case where 7 is a closed curve in M 2 which lies entirely in the closed 
upper half-plane, and which is tangent to second order to the yi-axis at the origin. We 
now write some neighbourhood of G Y as a horizontal graph over the (y\, x)-plane, i.e. 
Y = {1J2 = u(y%,x)} for \y\\ < 5, x < 5. Set s = y± for simplicity. This function satisfies 
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the minimal surface equation, which in this coordinate system takes the form 



2(1 4- u 2 + it 2 ) 

F(u) = (1 + u 2 x )u ss - 2u s u x u sx + (1 + u 2 s )u xx - -u x = 0. (7.28) 

x 

(Note that this is simply the specialization of (|2.6p when the base of the cylinder T is 
flat, so ft = and w = 1.) Furthermore, since Y lies to one side of the (s, x) plane, u > 
everywhere, and it has the asymptotic expansion 

u(s, x) ~ a(s)x 2 + 0(x 3+a ). (7.29) 

The absence of the x 3 term is because Y is critical for A; furthermore, o(0) is the one 
half the curvature of 7 at 0, and hence because it osculates the line there, a(0) = 0. 

Now choose any j3 6 (0, a) and define u c = u — cx 3+/3 for c > 0. By the various 
properties above, if we fix 5 then choose c sufficiently small, the function u c > on all 
four sides of the rectangle \s\ < 5, < x < 5. However, using (|7.29p and the fact that 
[3 < a, we also have that u c (0,x) < for x sufficiently small. This means that the 
minimum of u c is achieved somewhere strictly inside this rectangle, and of course at that 
point (s c ,x c ), we have F(u c )(s c , x c ) > 0. 

On the other hand, we compute that 

F{u - cx 3+/3 ) = F(u) - L u (cx i+I3 ) + 0(cx 6+2/3 ); 

the first term on the right vanishes, while the operator in the second term is the lin- 
earization of J- at u, 

L u v = (1 + u 2 x )v ss - 2u s u x v sx + (1 + u 2 )v xx + 2(u x u ss - u s u sx )v x + v s 

2(1 + u 2 x + u 2 s ) Au 2 Au s u x 

-\-ZyU s U xx U X U SX )V S V x V x V s . 

X XX 

By (|7.29p and scale-invariant Schauder estimates, rr _2 |u| + x _1 |Vu| + |V 2 u| < K. Hence 
altogether 

F(u c ) < -c/3(3 + f3)x 1+p + Kcx 2+f3 , 

where K is independent of c and 5. Choosing 5 sufficiently small, we can ensure that 
TiyLc) < everywhere in this box, which contradicts that it is positive at (s c ,x c ). This 
proves the theorem. □ 



8 Connections with the Willmore functional 

It is not particularly surprising that the functional A is connected with the Willmore 
functional, which by definition is the total integral of the square of mean curvature. In 
this final section we explore some of these relationships. 

Fixing a special bdf x, then the expansion of g = x 2 g has only even powers, so 
its natural extension to a Z2-invariant metric on the double of M across its boundary 
is smooth; furthermore, any surface Y 6 M^(M) can also be doubled to a closed C 2,1 
surface. We denote these doubles by 2M and 2Y, respectively. 

Proposition 8.1. IfY £ M k (M), then 

Ai Y) = ~W(2Y). 
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Proof. We have already noted that \k\ 2 dA g = \k\ 2 dAg. Next, observe that 

= 2^H 2 - , 

where the the principal curvatures of Y with respect to g. Thus finally 

A(Y) = -2tt X (Y) - J y (h 2 - k{k 2 ) dAg 

-2vr X (y) ~\J (h 2 - k{K 2 ) dAg = -2tt X (Y) - \w(2Y) + 2vr X (y), 

since x@Y) = 2 X (Y). □ 

On the other hand, it does not seem to be the case that the extended functional 1Z 
on Mk(M) has any simple connection with the Willmore functional. Nonetheless, this 
result recasts the program of finding extremals of A in a different and more classical 
light. Let us conclude by explaining this a bit further. 

The conformal invariance of the Willmore functional makes its variational theory 
quite challenging. The existence of minimizers of the Willmore functional for closed 
surfaces in IR 3 with genus less than or equal to some fixed constant is proved in |37j . and 
this result is sharpened in [7], where it is shown how to prevent a 'drop of genus' in these 
minimization arguments. We have just shown that the problem of finding extrema, and 
in particular, maxima, for A on some given moduli space A4fc(M) is equivalent to finding 
constrained extrema (in particular, minima) of W on 2M with respect to the metric g~, 
within the restricted class of surfaces which are invariant under the Z 2 involution and 
which are minimal with respect to g. If it were possible to adapt the arguments from 
[37] to this ambiently curved setting, we could prove the existence of such extrema. This 
may well be subtle, and the strengthened result from [7] may not be available, even when 
M = H 3 . Indeed, consider that we have proved that there are no nondegenerate critical 
points for A on A4fc(H 3 ) when k > 0. This indicates that any extremizing sequence Yj, 
e.g. one for which A(Yj) tends to the supremum, probably does not converge to a surface 
of the same genus. 
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